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(Formally  Project  No.  9AE-0091.00) 


Use  of  an  Open  Systems  Approach  for  Weapon  Systems 


Executive  Summary 


Introduction.  This  report  discusses  the  extent  that  acquisition  program  managers 
considered  and  used  an  open  systems  approach  in  the  design  and  development  of  major 
defense  weapon  systems.  The  Under  Secretary  of  Defense  for  Acquisition, 

Technology,  and  Logistics  mandated  the  use  of  an  open  systems  approach  in  the 
acquisition  process  to  reduce  the  cost  of  ownership  of  weapons  systems  while 
increasing  interoperability  and  useful  life.  The  Under  Secretary  chartered  an  open 
systems  Joint  Task  Force  (the  Joint  Task  Force)  in  November  1994  to  facilitate  DoD 
use  and  implementation  of  an  open  systems  approach  in  weapon  systems  acquisition. 

Objectives.  The  primary  audit  objective  was  to  evaluate  the  extent  that  program 
managers  considered  and  used  an  open  systems  approach  in  weapons  systems 
development.  We  also  reviewed  the  effectiveness  of  management  controls  applicable  to 
the  audit  objective. 

Results.  The  Joint  Task  Force  has  worked  diligently  to  implement  the  open  system 
approach  for  DoD  weapons  systems.  However,  the  Joint  Task  Force  needed  increased 
assistance  from  the  Defense  and  Component  acquisition  executives,  as  well  as  program 
managers,  to  implement  the  use  of  an  open  systems  approach  in  the  systems  acquisition 
process. 

Of  the  17  major  Defense  acquisition  programs  that  gained  approval  to  begin  program 
definition  and  risk  reduction  or  to  enter  engineering  and  manufacturing  development 
between  March  1996  and  July  1999,  14  programs  proceeded  into  the  next  acquisition 
phase  without  program  mangers  clearly  defining  open  system  design  objectives  or 
strategy  for  achieving  the  objectives.  Specifically,  users  and  program  managers  did  not 
include  language  concerning  the  required  use  of  an  open  systems  design  in  acquisition 
planning  documents.  The  following  list  of  seven  acquisition  planning  documents  shows 
the  number  of  programs  that  did  not  include  language  concerning  the  required  use  of 
open  systems  out  of  the  number  of  programs  that  prepared  the  cited  document. 

•  operational  requirements  document  (6  of  17  programs), 

•  single  acquisition  management  plan  (2  of  12  programs), 

•  acquisition  plan  (3  of  5  programs), 

•  system  engineering  management  plan  (2  of  6  programs), 

•  request  for  proposal  (9  of  17  programs), 

•  contract  statement  of  work  (8  of  15  programs),  and 

•  test  and  evaluation  master  plan  (11  of  17  programs) 


As  a  result,  DoD  acquisition  managers  did  not  have  assurance  at  program  milestone 
reviews  that  program  managers  required  and  stressed  the  importance  of  implementing 
open  system  design  objectives  in  acquisition  strategies  to  weapon  systems  contractors 
(finding  A). 

Detailed  documentation  reviews  of  4  of  the  17  major  Defense  acquisition  program 
offices  showed  that  program  managers  for  3  of  the  4  programs  did  not  document  a 
means  for  determining  die  extent  of  design  openness  of  systems,  subsystems,  and 
components.  Also,  DoD  guidance  on  open  systems  did  not  require  program  managers 
to  assess  the  impact  of  a  given  level  of  design  opermess  on  the  long-term  viability  and 
affordability  of  systems.  Without  a  means  to  measure  the  progress  and  the  impact  of 
implementing  an  open  systems  approach,  acquisition  decision  makers  can  not  readily 
gauge  how  well  program  managers  are  achieving  the  advantages  of  using  an  open 
systems  design  approach  or  assessing  the  susceptibility  of  a  weapon  systems  design  to 
obsolescence  or  costly  upgrades  to  counter  foreign  military  threats  (finding  B). 

See  Appendix  A  for  details  on  the  management  control  program  as  it  relates  to  controls 
over  program  managers  considering  and  using  an  open  systems  design  approach  in  key 
acquisition  planning  documentation.  Recommendations  in  this  report,  if  implemented, 
will  improve  the  process  for  defining  and  documenting  open  systems  objectives  in  key 
acquisition  planning  documentation  and  correct  the  material  control  weakness  identified 
in  die  report. 

Summary  of  Recommendations.  We  recommend  that  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics  enforce  the  requirement  that  program 
managers  consider  and  use  open  systems  during  the  acquisition  milestone  review 
process  and  that  program  manager  progress  in  inserting  open  systems  design 
requirements  in  key  acquisition  planning  documents  is  measured.  We  also  recommend 
that  program  managers  be  required  to  include  open  systems  objectives  in  test  and 
evaluation  master  plans  and  to  assess  the  impact  of  projected  system  design  openness  to 
provide  acquisition  milestone  decision  makers  assurance  at  acquisition  milestone 
reviews  that  program  managers  had  stressed  the  iinportance  of  implementing  open 
systems  objectives  into  acquisition  strategies.  Additionally,  we  recommend  that  the 
Joint  Task  Force  provide  program  managers  with  general  templates  for  inserting  open 
systems  design  language  in  the  key  acquisition  planning  documents  and  provide 
guidance  to  h^elp  program  managers  document  die  means  for  determining  the  extent  of 
system  design  opermess. 

Management  Comments.  The  Director,  Operational  Test  and  Evaluation,  agreed  to 
support  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  in 
revising  acquisition  policy  to  require  program  managers  to  include  open  systems 
objectives  in  test  and  evaluation  master  plans.  The  Army  also  agreed  with  the 
recommendations  in  the  report. 

Audit  Response.  The  comments  from  the  Director,  Operational  Test  and  Evaluation, 
were  responsive.  The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  did  not  comment  on  the  draft  of  this  report.  A  discussion  of  management 
comments  is  in  the  Findings  section  of  the  report,  and  the  complete  text  is  in  the 
Management  Comments  section.  We  request  that  the  Under  Secretary  of  Defense 
provide  comments  to  the  recommendations  in  this  final  report  by  July  14,  2000. 


11 


Table  of  Contents 


Executive  Summary  i 

Introduction 

Background  1 

Objectives  3 

Findings 

A.  Addressing  Use  of  Open  Systems  Objectives  in  the  Weapon  System 

Acquisition  Process  4 

B.  Determining  the  Extent  of  System  Design  Openness  16 


Appendixes 

A.  Audit  Process 

Scope  22 

Mediodology  23 

Management  Control  Program  23 

Summary  of  Prior  Coverage  24 

B.  Definitions  of  Open  Systems  Terms  25 

C.  Public  Law  and  Government  Policy  28 

D.  Open  Systems  Joint  Task  Force  Education  and  Outreach  Initiatives  29 

E.  Inclusion  of  Open  Systems  Objectives  in  Acquisition  Planning 

Documents  31 

F.  Report  Distribution  34 

Management  Comments 

Director,  Operational  Test  and  Evaluation  Comments  37 

Assistant  Secretary  of  the  Army  (Acquisition,  Logistics,  and 
Technology)  Comments  38 

Army  Program  Executive  Office  for  Ground  Combat  and  Support 
Systems  Comments 


39 


Background 


Open  Systems  Approach.  This  report  discusses  the  extent  that  acquisition 
program  managers  considered  and  used  an  open  systems  approach  in  weapon 
systems  development.  An  open  systems  approach  to  weapon  system 
development  is  an  acquisition  reform  initiative  requiring  program  managers  to 
implement  open  specifications  for  interfaces  between  systems,  subsystems,  and 
components.  Industry  standards  boards  (national  and  international)  develop  or 
adopt  open  specifications  to  accommodate  insertion  of  new  technologies  into 
systems.  An  open  system  design  for  a  weapon  system  is  characterized  by: 

•  well  defined,  widely  used,  preferably  nonproprietary  interfaces  and 
protocols; 

•  use  of  interface  standards  developed  or  adapted  by  industry-recognized 
standards  bodies. 

•  definition  of  all  aspects  of  system  interfaces  to  facilitate  new  or 
additional  capabilities  for  a  wide  range  of  applications;  and 

•  explicit  provision  for  system  expansion  or  upgrade  through  the 
incorporation  of  additional  or  higher  performance  elements  with  minimal 
negative  impact  on  the  existing  system. 

DoD  use  of  an  open  systems  approach  will  reduce  the  cost  of  ownership  of 
weapons  systems,  delay  system  obsolescenee,  and  allow  fielding  superior 
warfighting  capability  more  quickly.  An  open  systems  approach  reduces 
weapon  system  cost  through  facilitating  program  manager  use  of  widely 
accepted  standard  products  from  multiple  suppliers  in  DoD  weapon  systems.  If 
program  managers  define  weapon  system  architecture  by  specifications  and 
standards  used  in  the  private  sector,  DoD  can  leverage  the  benefits  of  the 
commercial  market  place,  and  take  advantage  of  the  competitive  pressures  that 
motivate  commercial  companies  to  improve  products  and  reduce  priees. 

Program  managers  can  then  have  access  to  alternative  sources  for  key 
subsystems  and  components  to  construct  DoD  weapon  systems.  The  open 
systems  approach  could  reduce  the  DoD  investment  early  in  the  weapon  system 
life  cycle  because  some  of  the  required  systems  or  components  may  be  available 
or  under  development  without  direct  DoD  investment.  Also,  program  managers 
can  competitively  select  production  sources  from  multiple  eompetitors. 
Additionally,  an  open  systems  approach  delays  system  obsolescence  by  allowing 
program  managers  to  incrementally  insert  technologieal  improvements  into 
existing  or  developing  systems  rather  than  having  to  make  large-scale  system 
redesigns  or  to  develop  new  systems.  Further,  an  open  systems  approach 
enables  program  managers  to  deliver  weapons  systems  to  warfighters  more 
quickly  as  a  result  of  reduced  developmental  effort.  Other  benefits  of  program 
managers  using  an  open  systems  approach  include: 
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•  greater  system  interoperability  for  more  effective  joint  and  allied  war 
fighting, 

•  closer  cooperation  between  commercial  and  military  electronics 
industries,  and 

•  better  international  competitiveness  of  the  U.S.  electronics  industry. 

Overall,  DoD  use  of  the  open  system  approach  should  help  in  pursuing  a 
focused  modernization  effort  to  maintain  a  qualitative  superiority  in  warfighting 
capabilities,  in  meeting  the  combat  forces  needs,  in  controlling  cost  growtii 
increases,  and  in  helping  DoD  to  meet  its  goals  and  performance  measures. 
Appendix  A  provides  details  on  DoD  goals  and  performance  measures  in 
response  to  the  Government  Performance  and  Results  Act  that  are  pertinent  to 
this  report.  Appendix  B  provides  a  listing  of  terms  and  definitions  germane  to 
understanding  program  manager  implementation  of  an  open  systems  approach  in 
designing  weapon  systems  architecture. 

Public  Law  and  Government  Policy.  Public  law  and  Government  acquisition 
policy  mandate  that  program  managers  consider  and  use  an  open  systems 
approach  in  the  weapon  system  acquisition  process.  Program  managers  must 
consider  and  use  open  systems  both  in  developing  new  weapon  systems  and  in 
modifying  existing,  or  legacy,  systems.  Appendix  C  provides  details  on 
relevant  public  law  and  Government  acquisition  policy  concerning  the  use  of  an 
open  systems  approach. 

Open  Systems  Joint  Task  Force.  The  Under  Secretary  of  Defense  for 
Acquisition  and  Technology  (now  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics)  chartered  an  open  systems  Joint  Task  Force  (the 
Joint  Task  Force)  in  November  1994  to  facilitate  DoD  use  and  implementation 
of  an  open  systems  approach  in  weapon  systems  acquisition.  Specifically,  the 
Under  Secretary  chartered  the  Joint  Task  Force  to  sponsor  and  accelerate  the 
adoption  of  open  systems  in  weapon  systems  and  subsystems  electronics  to 
reduce  life-cycle  cost  and  facilitate  effective  weapon  system  intra-  and 
interoperability.  In  executing  the  Under  Secretaries  charter,  the  Joint  Task 
Force  has  promoted  program  managers’  implementation  of  open  systems  policy, 
identified  opportunities  for  implementing  an  open  systems  approach,  developed 
training  and  education  programs,  and  coordinated  the  identification  and  selection 
of  open  systems  specifications  and  standards.  Appendix  D  provides  an 
overview  of  the  completed  and  ongoing  initiatives  of  the  Joint  Task  Force. 
AlAough  the  original  charter  for  Ae  Joint  Task  Force  expired  in 
November  1998,  the  Under  Secretary  provided  funding  to  support  the 
continuation  of  hie  Joint  Task  Force’s  efforts  for  an  additional  3  years  in 
June  1998. 
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Objectives 


The  primary  audit  objective  was  to  evaluate  the  extent  that  program  managers 
considered  and  used  an  open  systems  approach  in  weapons  systems 
development.  We  also  evaluated  the  management  controls  related  to  the  audit 
objective.  Appendix  A  discusses  the  audit  scope  and  methodology,  as  well  as 
management  controls  and  prior  audit  coverage. 
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A.  Addressing  Use  of  Open  Systems 
Objectives  in  the  Weapon  System 
Acquisition  Process 

The  DoD  acquisition  community  has  not  fully  applied  the  use  of  open 
systems  objectives  in  the  acq^uisition  planning  and  review  process.  Of 
the  17  major  Defense  acquisition  programs  that  gained  approval  to  begin 
program  definition  and  risk  reduction  or  to  enter  engineering  and 
manufacturing  development  between  March  1996  and  July  1999, 

14  programs  proceeded  into  the  next  acquisition  phase  without  program 
managers  clearly  defining  open  system  design  objectives  for  the  system 
and  the  strategy  for  achieving  the  objectives  in  key  acquisition  planning 
documents.  The  DoD  and  Component  acquisition  executives  allowed 
this  condition  to  occur  because  they  did  not  enforce  the  requirement  that 
program  managers  use  an  open  systems  design  approach  in  key 
acquisition  documents  as  part  of  the  acquisition  milestone  review 
process.  Without  acquisition  executive  enforcement,  program  offices  did 
not  aggressively  seek  guidance  and  training  in  using  an  open  systems 
approach  for  their  programs.  With  respect  to  training,  the  Joint  Task 
Force  had  emphasized  providing  open  system  guidance  and  training  to  a 
number  of  individual  acquisition  programs  but  had  only  provided  general 
guidance  to  the  broader  acquisition  community  through  die  Defense 
Acquisition  Deskbook,  seminars,  and  publications.  As  a  result,  DoD 
acquisition  decision  makers  did  not  have  assurance  at  program  milestone 
reviews  that  program  managers  required  and  stressed  the  importance  of 
implementing  open  system  design  objectives  in  their  acquisition 
strategies  to  weapon  system  contractors. 

Open  Systems  Policy 


The  Under  Secretary  of  Defense  for  Acquisition  and  Technology  (the  Under 
Secretary)  first  mandated  program  manager  use  of  an  open  systems  approach  in 
his  November  29,  1994,  memorandum,  “Acquisition  of  Weapons  Systems 
Electronics  Using  Open  Systems  Specifications  and  Standards.”  The 
memorandum  required  DoD  Components  to  use  open  systems  specifications 
(electrical,  mechanical,  thermal)  for  acquisition  of  weapon  systems  electronics 
to  the  greatest  extent  practical  in  an  effort  to  reduce  life-cycle  cost  and  facilitate 
effective  weapon  system  intra-  and  inter-operability.  In  DoD  Regulation 
5000. 2-R,  “Mandatory  Procedures  for  Major  Defense  Acquisition  Programs 
(MDAPS)  and  Major  Automated  Information  Systems  (MAIS)  Acquisition 
Programs,”  March  15,  1996,  the  Under  Secretary  expanded  the  requirement  for 
use  of  an  open  systems  approach  in  developing  systems  to  all  system  elements 
(mechanical,  electrical,  and  software).  The  Under  Secretary  provided  additional 
mandatory  policy  to  program  office  use  of  open  systems  design  in  Change  3, 
March  23,  1998,  and  Change  4,  May  11,  1999,  to  DoD  Regulation  5000. 2-R. 
Change  3  added  open  systems  to  the  essential  elements  that  program  managers 
must  address  in  their  acquisition  strategies.  Change  4  required  program 
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managers  to  establish  open  systems  objectives;  to  document  their  approach  to 
specifying  the  level(s)  of  openness  of  systems,  subsystems,  and  or  components 
to  be  acquired;  and  to  document  the  means  for  determining  the  extent  of 
opeimess  of  systems,  subsystems,  and  components. 

Recognizing  the  need  for  high-level  management  attention  to  program  use  of 
open  systems,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  issued  the  memorandum,  “Open  Systems  Acquisition  of  Weapon 
Systems,”  July  10,  1996,  directing  that  DoD  and  Component  acquisition 
executives  ensure  that  program  managers  include  open  systems  plans  in  the 
documentation  provided  to  support  acquisition  milestone  decisions.  The 
memorandum  further  directed  that  the  overarching  integrated  product  teams 
supporting  the  acquisition  executives  include  open  systems  planning  as  a  specific 
item  in  their  oversight  and  review  discussions  on  acquisition  programs. 
Appendix  C  provides  more  detailed  information  on  DoD  open  systems  policy  as 
well  as  information  on  public  law  concerning  the  use  of  open  systems. 

Defining  Open  Systems  Objectives 


We  reviewed  acquisition  planning  documents  for  17  major  Defense  acquisition 
programs  that  gained  approval  to  begin  program  definition  and  risk  reduction 
(Milestone  I),  or  to  enter  engineering  and  manufacturing  development 
(Milestone  II),  between  March  1996  and  July  1999.  DoD  and  Component 
acquisition  executives  allowed  14  of  the  17  major  Defense  acquisition  programs 
to  proceed  to  the  next  acquisition  phase  without  requiring  program  managers  to 
clearly  define  the  open  system  objectives  for  the  systems  or  the  strategy  for 
achieving  the  objectives.  Specifically,  users  and  program  managers  did  not 
include  language  concerning  the  required  use  of  an  open  systems  design  in 
acquisition  planning  documents.  The  following  list  of  seven  acquisition 
planning  documents  shows  the  number  of  programs  that  did  not  include 
language  concerning  the  required  use  of  open  systems  out  of  the  number  of 
programs  that  prepared  the  cited  document: 

•  operational  requirements  document  (6  of  17  programs), 

•  single  acquisition  management  plan  (2  of  12  programs), 

•  acquisition  plan  (3  of  5  programs), 

•  system  engineering  management  plan  (2  of  6  programs), 

•  request  for  proposal  (9  of  17  programs), 

•  contract  statement  of  work  (8  of  15  programs),  and 

•  test  and  evaluation  master  plan  (11  of  17  programs). 

Appendix  E  provides  details  on  the  inclusion  of  open  system  requirements  in 
acquisition  planning  documents  for  the  17  acquisition  programs  reviewed  and 
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indicates  which  programs  were  new  weapon  system  acquisitions  and  which 
programs  were  upgrades  to  existing,  or  legacy,  weapon  systems.  The  Joint 
Task  Force  emphasized  that  the  degree  that  program  managers  could  implement 
an  open  systems  design  for  legacy  systems  may  be  limited  because  of 
modifications  to  an  existing  weapon  system  design  that  may  not  be  open. 

Including  Open  Systems  Objectives  in  Acquisition  Planning 


Acquisition  planning  documents  serve  as  a  roadmap  to  program  managers  and 
contractors  for  program  execution  from  program  initiation  through 
postproduction  support.  Therefore,  the  DoD  acquisition  community,  in 
coordination  with  the  Joint  Chiefs  of  Staff  and  supporting  organizations  involved 
in  the  weapons  systems  requirements  generation  process,  must  include  open 
systems  requirements  in  acquisition  planning  documents  to  maximize  DoD 
effectiveness  in  implementing  an  open  systems  approach  for  developing  and 
acquiring  weapon  systems.  The  following  discusses  the  general  purpose  of  the 
seven  acquisition  planning  documents  that  the  Joint  Task  Force  agreed  should 
include  requirements  for  an  open  systems  design  approach  and  provides  the 
results  of  our  review  and  examples  of  how  program  managers  documented  the 
required  use  of  an  open  systems  approach. 

•  Operational  Requirements  Document.  The  operational  requirements 
document  is  a  product  of  the  requirements  generation  system.  The 
operational  requirements  document  is  a  formatted  statement  containing 
operational  performance  parameters  for  the  proposed  concept  or  system. 
Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3170.01  A,  “Requirements 
Generation  System,”  August  10,  1999,  requires  that  the  operational 
requirements  document  contain  the  performance  and  related  operational 
parameters  as  well  as  program  affordability  estimates  a  weapon  system 
acquisition  program  must  meet.  The  Instruction  promotes  program  use  of  the 
open  systems  approach  by  having  requirements  and  acquisition  planners  to 
include  a  description  of  the  benefits  of  evolutionary  acquisition  in  the 
operational  requirements  document  to  match  projected  threats. 

While  users  for  6  of  the  17  programs  did  not  include  open  systems  related 
language  in  their  operational  requirements  documents,  the  Operational 
Requirements  Document  for  the  National  Missile  Defense  Program, 

March  10,  1997,  clearly  defined  requirements  for  the  program  manager  to 
use  an  open  systems  approach.  Specifically,  the  operational  requirements 
document  required  the  program  manager  to  comply  with  an  established  open 
systems  architecture  and  to  establish  and  enforce  a  coherent  set  of  common 
open  technical  standards  across  missions  and  systems.  Additionally,  the 
operational  requirements  document  stated  that  the  program  manager  was  to 
maximize  the  use  of  standard  parts  and  components  already  in  the  military 
supply  system.  Further,  the  operational  requirements  document  stated  that 
the  program  manager  must  ensure  standardization,  interoperability,  and 
commonality  within  the  system  to  ensure  interoperability  with  other  systems 
and  to  reduce  the  cost  of  system  ownership. 
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•  Single  Acquisition  Management  Plan.  In  his  memorandum, 

“Reengineering  the  Acquisition  Oversight  and  Review  Process,”  April  28, 
1995,  the  Under  Secretary  allowed  program  mangers  to  consolidate 
milestone  documentation  in  a  single  acquisition  management  plan.  The 
single  acquisition  management  plan  provides  a  concise,  integrated 
description  of  the  overtdl  acquisition  and  program  management  strategy  and 
provides  a  vehicle  for  obtaining  the  statutory  and  regulatory  approvals 
required  for  a  program  manger  to  implement  the  acquisition  program. 
Aldiough  the  Under  Secretary  did  not  mandate  a  specified  format  for  the 
single  acquisition  management  plan,  the  plan  normally  includes  a  program 
summary,  a  mission  description,  the  acquisition  strategy,  the  engineering 
and  tecluiical  approach,  and  the  program  support  strategy  .  Program 
managers  who  use  a  single  acquisition  management  plan  in  place  of  separate 
acquisition  documents  for  the  acquisition  plan  and  the  engineering 
management  plan,  should  include  specific  open  systems  language  in  the 
acquisition  strategy  and  systems  engineering  sections  of  the  plan  to 
document  that  the  contractor  will  be  required  to  use  an  open  systems 
acquisition  and  management  approach. 

Program  managers  for  2  of  the  12  programs  did  not  include  open  systems 
related  language  in  their  single  acquisition  management  plans.  However,  the 
Air  Force  Space  Based  Infrared  System  Single  Acquisition  Management 
Plan,  October  1,  1996,  strongly  emphasized  the  Air  Force’s  commitment  to 
open  systems  in  the  engineering  and  manufacturing  design  of  the  system.  In 
the  single  acquisition  management  plan,  the  program  manager  stated  that  he 
would  maintain  a  systems  engineering  process  over  the  entire  program  to 
include  use  of  an  open  systems  concept.  Further,  the  program  manager 
stated  in  the  plan  that  he  would  add  incentives  to  an  open  system  approach 
by  encouraging  the  contractor  to  implement  an  architecture  diat  defined 
internal  system  interfaces  by  using  open  standards  that  industry  had  adapted. 

•  Acquisition  Plan.  The  Defense  Federal  Acquisition  Regulation  Supplement, 
Part  207,  “Acquisition  Planning,”  October  1,  1999,  requires  program 
managers  to  prepare  written  acquisition  plans  for  weapon  system  acquisitions 
when  contract  costs  are  expected  to  total  $5  million  or  more.  In  the 
acquisition  plan,  the  Supplement  requires  program  managers  to  include 
information  on  acquisition  background,  objectives,  and  a  plan  of  action  for 
the  development  effort  that  addresses  product  descriptions,  logistics 
considerations,  budget  and  funding,  and  other  program  considerations.  The 
program  manager  provides  the  acquisition  plan  to  the  contract  administration 
organization  to  facilitate  resource  allocation  and  planning  for  evaluating  and 
managing  contractor  risk.  Program  manager  inclusion  of  open  systems 
objectives  in  the  acquisition  plan  is  essential  for  ensuring  that  assigned 
contract  administration  organizations  evaluate  contractor  implementation  of 
an  open  systems  approach  in  the  design  of  the  weapon  system. 

Program  managers  for  3  of  the  5  programs  did  not  include  open  systems 
related  language  in  their  acquisition  plans.  The  program  manager  for  the 
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United  States  Marine  Corps  H-1  Upgrade  Program  issued  an  acquisition  plan 
on  September  27,  1996,  that  clearly  documented  the  program  manager’s 
plans  for  using  an  open  systems  approach.  In  the  acquisition  plan,  Sie 
program  manager  specified  that  the  contractor  would  use  an  open  systems 
approach  that  entailed  contractor  use  of  commercial  interface  standards  to 
ensure  form,  fit,  and  function  interchangeability,  both  internal  and  external, 
to  the  components  that  comprise  the  cockpit  upgrade.  Further,  the  program 
manager  stated  that  the  contractor  through  the  system  design  effort  must 
allow  for  future  modifications  and  system  integration  flexibility  to  reduce  the 
overall  lifecycle  cost  of  the  helicopter  cockpit  with  particular  emphasis  on 
reducing  system  operations  and  support  costs. 

•  Systems  Engineering  Management  Plan.  The  Defense  Acquisition 
Deskbook  Critical  Process  Assessment  Tool,  “Systems  Engineering,” 

August  14,  1998,  defines  the  systems  engineering  management  plan  as  the 
program  plan  for  conducting  systems  engineering.  The  Deskbook  states  that 
the  program  office  may  need  the  systems  engineering  management  plan  to 
understand  the  contractors  system  engineering  process  well  enough  to 
maintain  insight  into  its  progress.  The  program  manager’s  inclusion  of  open 
systems  objectives  in  this  document  is  important  because  the  contractor 
needs  to  focus  on  design  flexibility  and  open  interface  selection  to  adapt  to 
technology  evolution  and  enable  system  upgrades  over  time. 

Program  managers  for  2  of  the  6  programs  did  not  include  open  systems 
related  language  in  their  system  engineering  management  plans.  TTie  Navy 
Theater  Wide  Theater  Ballistic  Missile  Program  Systems  Engineering 
Management  Plan,  March  22,  1999,  did  include  open  systems  related 
language.  In  the  plan,  the  program  manager  stated  that  the  program  office 
engineering  staff  would  ensure  that  the  contractor  used  an  open  systems 
architecture  to  the  maximum  extent  possible  to  simplify  the  contractor 
making  modifications  to  the  subsystems  and  to  enhance  interoperability  with 
other  ttieater  missile  defense  and  theater  missile  defense  support  systems. 

•  Request  for  Proposal.  The  Federal  Acquisition  Regulation, 

Subpart  15.203,  “Requests  for  Proposal,”  October  1,  1999,  requires 
contracting  officers  for  negotiated  acquisitions  to  use  requests  for  proposals 
to  communicate  Government  requirements  to  perspective  contractors  and  to 
solicit  contractor  proposals.  In  three  sections  of  the  request  for  proposal, 
contracting  officers  can  give  contractors  information  needed  to  develop 
contract  proposals  that  will  effectively  implement  an  open  systems  approach. 
The  tiiree  sections  are  Section  L,  “Instructions,  Conditions,  and  Notices  to 
Offerors  or  Quoters;”  Section  M,  “Evaluation  Factors  for  Award;”  and  the 
statement  of  objectives.  Through  these  proposal  sections,  the  contracting 
officer  can  advise  prospective  contract  offerors  that  they  will  be  required  to 
develop  a  system  using  an  open  systems  approach  and  Aat  their  proposal 
must  address  an  open  systems  concept  if  they  want  to  be  considered  as  a 
responsive  offeror  to  the  request  for  proposal. 

Program  managers  for  9  of  the  17  programs  did  not  include  open  systems 
related  language  in  the  request  for  proposal.  An  example  of  a  request  for 
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proposal  that  included  open  systems  related  language  is  the  engineering  and 
manufacturing  development  contract  for  the  Air  Force  Global  Broadcast 
Satellite  Program.  The  request  for  proposal  gave  offerors  clear  notice  on 
instructions  and  evaluation  factors  pertaining  to  using  an  open  systems 
approach  in  the  design  and  development  of  die  satellite. 

□  Section  L  of  the  request  for  proposal  requires  that  offerors  identify 
contractor  efforts  to  ensure  that  use  of  open  standards  and  open 
architectures  will  be  a  driving  factor  in  Ae  contractor’s  system 
design  approach.  Section  L  also  requires  offerors  to  describe  system 
segment  in  terms  of  functions  and  interfaces  and  show  how  open 
systems  architectures  and  standards  apply  to  the  architecture. 

Further,  Section  L  requires  offerors  to  describe  plans  for  evolving 
the  proposed  system  architecture  to  meet  system  interim,  threshold, 
and  objective  performance  capabilities. 

□  Section  M  of  the  request  for  proposal  identifies  the  factors  that  the 
contracting  officer  will  consider  in  awarding  the  contract.  Section  M 
states  that  the  contracting  officer  will  consider  the  offeror’s  use  of  an 
open  systems  design  in  Ae  technical  evaluation  factor  assessment 
used  to  make  the  decision  to  award  the  engineering  and 
manufacturing  development  contract. 

□  Offerors  use  the  statement  of  objectives  to  develop  their  proposed 
contract  statement  of  work  that  supports  and  defines  their  proposed 
efforts.  The  statement  of  objectives  states  that  the  contractor  should 
maximize  use  of  commercial  equipment  and  software,  open  systems, 
and  use  of  open  nonproprietary  network  and  communications 
protocols  and  standards.  Additionally,  the  statement  of  objectives 
states  that  the  contractor  should  design  the  system  architecture  to 
accommodate  evolutionary  hardware  and  software  changes  to  achieve 
future  system  requirements. 

•  Contract  Statement  of  Work.  The  Federal  Acquisition  Regulation, 
Subparts  15.406-1,  “Uniform  Contract  Format,”  and  15.406-2,  “Part  1  - 
The  Schedule,”  October  1,  1995,  require  Agency  solicitations  for 
contracts  to  include  a  statement  of  work,  or  other  description  that  defines 
the  Government’s  requirements.  Program  manager  inclusion  of  open 
systems  objectives  in  this  document  is  necessary  to  ensure  that  the 
contractor  uses  an  open  systems  design  approach  in  the  system’s  design. 
Program  managers  can  also  use  provisions  in  the  contract  statement  of 
work,  along  with  the  contract  data  requirements  list,  to  require  the 
contractor  to  provide  the  program  manager  with  information  to  identify 
the  extent  of  system  design  openness. 

Program  managers  for  8  of  the  15  programs  did  not  include  open 
systems  related  language  in  their  contract  statements  of  work.  A  positive 
example  is  the  Army’s  statement  of  work  in  the  contract  for  the  Guided 
Multiple  Launch  Rocket  System  that  defined  the  required  level  of  system 
openness  as  subsystem  interfaces.  The  statement  of  work  also  required 
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that  the  contractor  describe  subsystem  interfaces  in  interface  control 
documents  and  to  list  and  justify  all  proprietary  [closed]  interfaces, 
including  proprietary  extensions  to  open  standards.  Further,  the 
statement  of  work  required  that  the  contractor  identify,  at  systems  design 
reviews,  all  configuration  items  which,  as  a  result  of  the  proposed  open 
systems  architecture  design,  are  amenable  to  efficient  tectoology  and  or 
supplier  insertion  at  any  stage  during  system  engineering  and 
manufacturing  development  and  system  production. 

•  Test  and  Evaluation  Master  Plan.  DoD  Regulation  5000. 2-R  requires 
that  program  managers  prepare  a  test  and  evaluation  master  plan  to 
provide  a  framework  within  which  to  generate  detailed  test  and 
evaluation  plans  and  identifies  necessary  developmental  and  operational 
test  and  evaluation  activities  for  the  weapon  system.  The  program 
manager’s  inclusion  of  open  systems  objectives  in  this  document  is 
needed  to  ensure  that  developmental  and  operational  test  organizations 
verify  contractor  insertion  of  an  open  systems  design  through  planned 
testing  of  system  components. 

Program  managers  for  11  of  the  17  programs  did  not  include  open 
systems  related  language  in  their  test  and  evaluation  master  plans,  the 
Test  and  Evaluation  Master  Plan  for  the  Army  CH-47  Improved  Cargo 
Helicopter,  September  29,  1998,  did  include  open  systems  related 
language.  The  plan  stated  that  the  contractor  would  select  or  develop  all 
hardware  and  software  for  the  helicopter  cockpit  within  the  concepts  of 
an  open  systems  design.  Further,  the  plan  established  compatibility  of 
system  electronic  architecture  wi&  the  Joint  Technical  Architecture  - 
Army  as  a  critical  technical  parameter  for  the  helicopter  cockpit 
program.  Accordingly,  the  plan  specified  that  the  contractor  must 
successfully  demonstrate  compatibility  with  the  Joint  Technical 
Architecture  -  Army  during  production  qualification  tests.  The  Joint 
Technical  Architecture  -  Army  includes  commercial  standards  to  provide 
program  managers  with  building  codes  for  the  development  of  all  Army 
programs. 

Use  of  Open  Systems  Objectives  in  the  Acquisition  Planning 
and  Review  Process 


Program  managers  did  not  routinely  include  open  systems  design  objectives  in 
acquisition  planning  and  review  because  the  Defense  and  Component  acquisition 
executives  did  not  enforce  the  requirement  that  program  managers  use  an  open 
systems  design  approach  in  key  acquisition  documents  as  part  of  the  acquisition 
milestone  review  process.  Without  acquisition  executive  enforcement,  program 
offices  did  not  always  aggressively  seek  guidance  and  training  in  using  an  open 
systems  design  approach  for  their  programs.  With  respect  to  training,  the  Joint 
Task  Force  had  emphasized  providing  detailed  guidance  and  training  to 
designated  pilot  programs  and  to  those  program  managers  that  sought  advice. 
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while  relying  on  the  efforts  of  the  acquisition  executives  and  advisory  guidance 
in  the  Defense  Acquisition  Deskbook,  conferences,  and  a  web  site  to  provide 
guidance  and  training  to  the  larger  acquisition  community. 

Acquisition  Milestone  Review  Process.  The  Defense  and  Component 
acquisition  executives  had  not  implemented  effective  controls  to  enforce  the 
requirement  that  program  managers  use  an  open  systems  design  approach  in  key 
acquisition  documents  as  part  of  the  acquisition  milestone  review  process. 
Because  the  DoD  and  Component  overarching  integrated  product  teams 
supporting  the  acquisition  executives  did  not  routinely  include  prograni  manager 
implementation  of  open  systems  in  their  reviews,  program  managers  did  not 
always  aggressively  seek  guidance  and  training  in  using  an  open  systems 
approach.  To  determine  if  overarching  integrated  product  teams  were 
addressing  the  use  of  open  systems  during  the  oversight  and  review  process,  we 
reviewed  team  reports  for  12  of  the  17  major  Defense  acquisition  programs 
included  in  our  audit  scope.  In  11  of  the  12  team  reports,  the  teams  did  not 
mention  a  discussion  with  program  managers  concerning  the  use  of  an  open 
systems  design  approach. 

To  remedy  the  above  condition,  the  Under  Secretary  needs  to  establish 
performance  goals  and  metrics  to  measure  program  manager  progress  in 
inserting  open  systems  design  requirements  in  key  acquisition  documents  in 
support  of  acquisition  milestone  reviews.  The  overarching  integrated  product 
teams  are  in  the  best  position  to  measure  program  manager  performance  against 
established  performance  goals  and  metrics  as  part  of  the  milestone  review 
process.  An  open  systems  performance  goals  and  metrics  would  also  ensure 
that  the  overarching  integrated  product  teams  supporting  the  milestone  decision 
authorities  include  open  systems  planning  as  a  specific  item  in  their  oversight 
and  review  discussions  on  acquisition  programs. 

The  Joint  Task  Force  indicated  that  it  understands  the  need  for  performance 
goals  and  metrics  relating  to  program  manager  compliance  with  open  systems 
requirements.  As  to  whether  program  managers  addressed  open  systems  in  key 
acquisition  planning  documents,  the  Joint  Task  Force  stated  that  any  open 
systems  performance  metric  would  be  binary  in  nature  (yes  or  no).  In  addition 
to  addressing  open  systems  in  the  key  seven  acquisition  planning  documents 
previously  discussed,  the  Joint  Task  Force  stated  that  program  manager 
completion  of  market  research  on  the  availability  and  affordability  of 
commercial  interface  standards  and  system  architecture  studies  would  provide 
evidence  that  a  program  manager  had  used  an  open  system  approach. 

Guidance  and  Training  for  Pilot  Programs  and  Programs  Seeking  Advice. 
The  Joint  Task  Force  has  provided  extensive  guidance  and  training  on 
implementing  an  open  systems  design  approach  to  the  Navy  AV-8B  Aircraft 
Open  Systems  Core  Avionics  and  the  Air  Force’s  F-15E  Multipurpose  Display 
Processor  program  offices.  The  Principle  Deputy  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics  designated  the  two  acquisition 
programs  as  pilot  programs  for  implementing  an  open  systems  approach  on 
February  15,  1996.  Accordingly,  the  Joint  Task  Force  worked  directly  with  the 
pilot  program  staffs  in: 
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•  developing  open  systems  strategies, 

•  applying  open  systems  concepts  to  their  programs,  and 

•  providing  funding  to  assist  the  programs  offices  in  implementing  an 
open  systems  approach. 

The  Joint  Task  Force  also  provided  advice  and  assistance  on  an  open  systems 
design  approach  to  program  office  for  another  26  acquisition  programs  through 
collaboration  with  program  office  integrated  product  teams. 

Guidance  and  Training  for  the  General  Acquisition  Community.  The  Joint 
Task  Force  and  DoD  Components  provided  advisory  guidance  to  Ae  acquisition 
conmiunity  on  the  use  of  an  open  systems  design  approach  through  the  Defense 
Acquisition  Deskbook.  As  of  January  31,  2000,  the  Defense  Acquisition 
Deskbook  listed  304  documents  referencing  open  systems.  As  discussed  in 
Appendix  D,  the  Joint  Task  Force  has  also  developed  seminars,  mtorials,  and  a 
web  site  that  program  managers  can  access  to  obtain  information  on 
implementing  an  open  systems  design  approach.  While  available  guidance 
contains  constructive  information  on  implementing  an  open  systems  approach, 
the  Joint  Task  Force  acknowledged  that  it  could  provide  additional  clarifying 
guidance.  Specifically,  the  Joint  Task  Force  agreed  that  it  could  enhance  the 
understanding  and  effectiveness  of  program  managers  in  implementing  an  open 
systems  strategy  by  providing  guidance  templates  illustrating  how  program 
managers  can  address  open  systems  design  requirements  in  key  acquisition 
planning  documents  and  by  providing  open  systems  guidance  tailored  to  each 
acquisition  phase. 

Guidance  Templates.  The  Office  of  the  Director,  Joint  Staff,  in 
coordination  with  the  Joint  Task  Force,  established  a  requirement  for  using  an 
open  systems  approach  in  universal  guidance  templates  used  by  the  requirements 
community  to  prepare  the  operational  requirements  document.  The  Joint  Task 
Force  also  developed  templates  of  open  systems  information  for  four  other  key 
acquisition  documents.  The  Joint  Task  Force  provided  the  templates  to  program 
managers  requesting  assistance. 

Operational  Requirements  Document  Template.  The  Joint 
Task  Force  provided  the  Director,  Joint  Staff,  with  suggested  revisions  on  open 
systems  information  included  in  the  template  for  the  operational  requirements 
document  in  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3170.01, 
“Requirements  Generation  System,”  June  13,  1997.  On  August  10,  1999,  the 
Director,  Joint  Staff,  issued  die  revised  Instruction,  designated  3170. OlA,  which 
included  changes  in  response  to  the  Joint  Task  Force  suggested  revisions.  In  the 
revised  template  for  the  operational  requirements  document,  the  Director,  Joint 
Staff,  included  the  following  points  relevant  to  an  open  systems  approach: 

•  described  the  benefits  of  evolutionary  acquisition  for  proposed 
system  (if  appropriate)  in  the  general  description  of  operational 
capability  section. 
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•  included  interoperability  as  an  example  of  a  system  performance 
parameter  in  the  capabilities  required  section  and  as  a  support 
objective  in  the  program  support  section, 

•  specified  that  the  system  must  comply  with  applicable  information 
technology  standards  contained  in  Ae  DoD  Joint  Technical 
Architecture,  and 

•  specified  that  the  system  cost  figure  should  be  stated  in  terms  of  a 
threshold  and  objective  to  provide  flexibility  to  allow  for  program 
evolution  in  the  program  affordability  section. 

The  revised  language  in  Instruction  3170. OlA  should  increase  the  emphasis 
weapon  systems  users  and  Service  Chiefs  of  Staff  give  to  open  systems  when 
processing  the  operational  requirements  document.  Also,  it  will  help  the 
acquisition  cormnunity  effectively  flow  an  open  systems  requirements  into  the 
acquisition  planning  documents  prepared  based  on  the  operational  requirements 
document. 


Templates  for  Other  Acquisition  Planning  Documents.  On 
request,  the  Joint  Task  Force  provided  program  office  staffs  with  template 
examples  showing  open  systems  language  that  can  be  used  in  the  single 
acquisition  management  plan,  the  systems  engineering  management  plan,  the 
request  for  proposal,  and  the  contract  statement  of  work.  However,  the  Joint 
Task  Force  had  not  provided  the  broader  acquisition  community  with  the  four 
document  template  examples  or  prepared  template  examples  showing  open 
systems  language  that  can  be  used  in  preparing  the  acquisition  plan  and  the  test 
and  evaluation  master  plan. 

The  availability  of  templates  of  open  systems  language  that  program  managers 
can  include  in  the  six  key  acquisition  planning  documents  would  greatly  assist 
program  offices  in  establishing  appropriate  open  systems  language  in  the 
acquisition  planning  documents.  Also,  the  Joint  Task  Force’s  development  and 
inclusion  of  the  document  templates  in  the  Defense  Acquisition  Deskbook  would 
be  a  timely  response  to  a  General  Accounting  Office  report  citing  the  need  for 
DoD  to  improve  acquisition  training.  The  General  Accounting  Office  criticized 
DoD  training  related  to  acquisition  reform  initiatives  in  GAO/NSIAD-99-206, 
“Best  Practices  -  DoD  Training  Can  Do  More  to  Help  Weapon  System 
Programs  Implement  Best  Practices,”  August  1999.  The  report  stated  that 
program  officials  reported  that  DoD  training  did  not  prepare  them  for 
implementing  revised  practices.  The  General  Accounting  Office  stated  that  the 
DoD  training  typically  provided  only  an  awareness  of  the  practices,  not  the 
knowledge  needed  for  actual  implementation.  The  template  examples  for  the  six 
key  acquisition  planning  documents  would  provide  program  offices  with  a 
practical  supplement  to  conceptual  training  on  implementing  an  open  systems 
design  approach. 
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Assurance  That  Open  Systems  Objectives  Were  Met 


As  a  result,  DoD  acquisition  decision  makers  did  not  have  assurance  at  program 
milestone  reviews  that  program  managers  required  and  stressed  the  importance 
of  open  systems  design  objectives  to  weapon  system  contractors.  Unless 
Defense  and  Component  acquisition  executives  fully  emphasize  the  importance 
of  maximizing  program  manger  and  contractor  use  of  an  open  systems  design 
approach  from  the  inception  of  acquisition  programs,  DoD  will  not  fully  realize 
the  long-term  benefits  of  using  an  open  systems  design  approach  to  develop  and 
acquire  weapon  system. 

Recommendations,  Management  Comments,  and  Audit 
Response 

A.l.  We  recommend  that  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics: 

a.  Enforce  the  requirement  for  overarching  integrated  product 
teams  within  OSD  and  DoD  components  to  assess  open  systems  planning  as 
a  specific  item  for  inclusion  in  the  acquisition  oversight  and  review  process 
when  preparing  for  system  milestone  reviews. 

b.  Establish  performance  goals  and  metrics  to  measure  program 
manager  progress  in  inserting  open  systems  design  requirements  in  key 
acquisition  planing  documents. 

Management  Comments.  The  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  did  not  comment  on  the  recommendation.  We 
request  that  the  Under  Secretary  of  Defense  provide  comments  to  the  final 
report. 

A.2.  We  recommend  that  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  and  the  Director,  Operational  Test  and 
Evaluation  revise  DoD  Regulation  5000. 2-R,  “Mandatory  Procedures  for 
Major  Defense  Acquisition  Programs  (MDAPs)  and  Major  Automated 
Information  Systems  (MAIS)  Acquisition  Programs,”  May  11,  1999,  to 
require  program  managers  to  include  open  systems  objecuves  in  test  and 
evaluation  master  plans  to  emphasize  to  developmental  testers  the  need  to 
verify  the  contractor’s  use  of  an  open  system  design  approach. 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Comments.  The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  did  not  comment  on  the  recommendation.  We  request  that  the  Under 
Secretary  of  Defense  provide  comments  to  the  final  report. 
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Director,  Operational  Test  and  Evaluation,  Comments.  The  Director, 
Operational  Test  and  Evaluation,  concurred,  stating  that  he  would  support  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  in 
making  the  recommended  revision  to  DoD  Regulation  5000. 2-R. 

Audit  Response.  The  Director,  Operational  Test  and  Evaluation,  comments 
were  responsive  to  the  recommendation.  We  request  that  the  Under  Secretary 
coordinate  with  the  Director,  Operational  Test  and  Evaluation,  and  provide 
comments  to  the  final  report. 

A. 3.  We  recommend  that  the  Director,  Open  Systems  Joint  Task  Force, 
include  in  the  Defense  Acquisition  Deskbook  suggested  general  template 
language  relating  to  program  manager  implementation  of  an  open  systems 
acquisition  strategy  in  the: 

a.  single  acquisition  management  plan, 

b.  acquisition  plan, 

c.  systems  engineering  management  plan, 

d.  request  for  proposal, 

e.  contract  statement  of  work,  and 

f.  test  and  evaluation  master  plan. 

Director,  Open  Systems  Joint  Task  Force  Comments.  The  Director,  Open 
Systems  Joint  Task  Force,  did  not  comment  on  the  recommendation.  We 
request  that  the  Director  provide  comments  to  the  final  report. 

Assistant  Secretary  of  the  Army  (Acquisition,  Logistics,  and  Technology) 
Comments.  Although  not  required  to  comment,  the  Acting  Deputy  Assistant 
Secretary  for  Plans,  Programs,  and  Policy,  Office  of  the  Assistant  Secretary  of 
the  Army  (Acquisition,  Logistics,  and  Technology),  agreed  with  the 
recommendations  in  the  draft  report.  Further,  the  Acting  Deputy  Assistant 
Secretary  stated  that  the  availability  of  general  template  language  will  benefit 
persoimel  who  have  the  task  of  drafting  the  acquisition  documents  discussed  in 
the  draft  report. 

Army  Program  Executive  Office  for  Ground  Combat  and  Support  Systems 
Comments.  Although  not  required  to  comment,  the  Program  Executive  Officer 
stated  that  he  agreed  with  the  draft  report. 
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B.  Determining  the  Extent  of  System 
Design  Openness 

Detailed  review  of  program  documentation  at  4  of  the  17  major  Defense 
acquisition  program  offices  showed  that  program  managers  for  3  of  the  4 
programs  did  not  document  a  means  for  determining  the  extent  of  design 
openness  of  systems,  subsystems,  and  components.  Additionally, 
guidance  on  open  systems  did  not  require  program  managers  to  assess 
die  impact  of  their  planned  level  of  design  openness  on  the  long-term 
viability  and  affordability  of  systems.  These  conditions  occurred  because 
the  Joint  Task  Force  did  not: 

•  provide  program  managers,  in  general,  with  guidance  on  how  to 
document  the  means  for  determining  the  extent  of  system  design 
openness;  and 

•  establish  acquisition  policy  to  recognize  that  determining  the  level 
of  openness  of  system  design  is  most  meaningful  when  combined 
with  program  impact  assessments. 

Without  a  means  to  measure  the  progress  and  the  impact  of 
implementing  an  open  systems  approach,  acquisition  decision  makers  can 
not  readily  gauge  how  well  program  managers  are  achieving  the 
advantages  of  using  an  open  system  design  approach  and  assess  the 
susceptibility  of  a  weapon  system  design  to  obsolescence  or  costly 
upgrade  to  counter  foreign  military  threats. 

Policy  for  Determining  the  Extent  of  System  Design  Openness 


The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  has 
recognized  the  need  to  determine  the  extent  of  openness  that  program  managers 
achieve  in  weapon  systems  design.  In  Change  3  to  DoD  Regulation  5000. 2-R, 
March  1998,  the  Under  Secretary  established  the  requirement  that  program 
managers  document  their  approach  for  measuring  the  level  of  openness  of 
systems,  subsystems,  and  components  to  be  acquired  and  devise  an  open 
systems  strategy  to  achieve  these  requirements.  In  Change  4  to  DoD  Regulation 
5000. 2-R,  May  1999,  the  Under  Secretary  modified  the  requirement  for 
measuring  openness  to  require  that  program  managers  document  their  means  for 
determining  the  extent  of  openness  of  system,  subsystems,  and  components 
assuring  conformance  to  open  standards  and  at  the  specified  levels  of  openness 
established  in  their  open  system  objectives. 
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Documenting  the  Means  for  Determining  the  Extent  of 
Openness  in  Weapons  Systems  Design 


Program  managers  for  3  of  the  4  program  offices  reviewed  (the  Army  Guided 
Multiple  Launch  Rocket  System,  the  Navy  and  Marine  Corps  UH-1  Helicopter 
Upgrade,  and  the  Air  Force  Global  Broadcast  Service)  did  not  document  a 
means  for  determining  the  extent  of  design  openness  in  systems,  subsystems, 
and  components.  The  program  manager  for  the  fourth  program  office  (the  Navy 
Tactical  Tomahawk)  requested  that  the  prime  contractor  provide  a  percentage 
measurement  of  the  level  of  design  openness  for  one  of  the  three  segments  of 
the  program. 

Guided  Multiple  Launch  Rocket  System.  Integrated  product  teams  for  the 
Guided  Multiple  Rocket  System  were  unsure  of  how  to  document  the  means  for 
determining  the  extent  of  systems,  subsystems,  and  components  design 
openness.  The  contract  statement  of  work  did  require  the  contractor  to  provide 
the  program  office  with  information  on  subsystem  interfaces  that  could  provide 
a  basis  for  determining  the  extent  of  an  open  system  design.  The  contract 
statement  of  work  established  interface  control  documents  as  the  means  for  the 
contractor  to  define  subsystems  interfaces  and  required  the  contractor  to  identify 
subsystem  interfaces  that  were  proprietary  in  nature.  However,  the  contractor 
had  not  yet  provided  the  program  office  with  any  interface  control  documents. 
The  integrated  product  teams  stated  that  the  contractor  would  provide  most 
interface  control  documents  for  approval  during  the  time  period  after  the 
preliminary  design  review  in  July  1999  and  before  the  critical  design  review 
scheduled  for  August  2000. 

H-1  Helicopter  Upgrade.  The  program  office  for  the  H-1  Helicopter  Upgrade 
also  was  unsure  of  how  to  document  the  means  for  determining  systems, 
subsystems,  and  components  design  openness.  The  program  office  stated  that 
the  concept  of  open  systems  was  new  and  that  not  enough  guidance  was 
available  for  them  to  fully  implement  open  systems  requirements.  The  program 
office  did  implement  an  open  systems  acquisition  strategy  for  the  avionics 
systems  but  not  for  the  propulsion  system  portion  of  the  upgrade.  The  program 
office  stated  that  it  would  be  difficult  to  determine  the  extent  of  openness  in  the 
avionics  design  because  the  contractor  was  awarded  a  streamlined  performance- 
based  contract.  The  contract  did  not  require  the  contractor  to  provide  visibility 
over  the  interface  control  documents.  The  program  office  added  that  their 
integrated  product  teams  would  have  some  insight  into  the  avionics  system 
design,  but  the  information  would  be  insufficient  to  make  a  periodic 
determination  on  the  extent  of  openness  of  the  avionics  system  design. 

Accordingly,  the  program  office  stated  that  it  would  not  be  able  to  readily 
determine  &e  extent  of  avionics  system  design  openness  although  it  believed  that 
its  acquisition  strategy  would  encourage  the  contractor  to  use  an  open  systems 
design  approach.  The  program  office  cited  two  factors  that  would  encourage 
the  contractor  to  use  an  open  systems  approach:  the  contractor,  as  part  of  die 
acquisition  strategy,  will  provide  operational  support  for  the  first  10  years  after 
the  helicopter  upgrade;  and  the  contract  statement  of  work  required  the 
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contractor  to  provide  for  an  easy  replacement  of  components  as  part  of  the 
system  reliability  requirement  for  mean-time-between-failure. 

While  the  acquisition  strategy  for  the  upgrade  program  should  result  in  a  system 
with  some  degree  of  openness,  the  program  office  has  no  assurance  that  the 
contractor  will  develop  systems  with  the  level  of  openness  that  the  Government 
desires.  Because  the  contractor  may  become  concerned  about  continued 
responsibility  for  maintaining  the  system  after  the  10-year  period,  the  contractor 
may  not  have  adequate  incentive  to  use  open  systems  interfaces.  Consequently, 
program  offices  should  not  rely  on  operational  requirements  and  contracting 
strategies  alone  to  achieve  an  open  systems  approach  and  should  take  contractual 
action  to  ensure  that  the  program  office  can  make  periodic  assessments  of  the 
level  of  opeimess  of  the  system  that  contractors  include  in  system  designs. 

Tactical  Tomahawk.  The  program  office  for  the  Tactical  Tomahawk  requested 
its  prime  contractor,  Raytheon-Hughes,  Tucson,  Arizona,  to  provide  a 
percentage  determination  of  the  level  of  design  openness  for  one  of  the  three 
segments  of  the  program:  the  Tactical  Tom^awk  missile,  the  weapons  control 
system,  and  the  mission  planning  system.  The  prime  contractor  provided  the 
program  office  with  a  percentage  on  the  level  of  design  openness  for  the  missile 
segment  through  assessing  the  number  of  common  and  commercial  interfaces  as 
well  as  software  transportability  and  modularity.  The  contractor  determined  that 
240  of  the  258  electrical  subsystem  interfaces  (93  percent)  in  the  missile 
segment  were  open  and  that  the  remaining  18  subsystem  interfaces  were 
proprietary  or  closed.  While  the  prime  contractor  provided  some  insight  into 
the  level  of  opeimess  for  the  missile  segment,  the  contractor  did  not  address 
software  and  mechanical  interfaces.  Since  July  1999,  the  program  office  has 
worked  with  the  missile  segment  contractor  to  modify  that  contract  with 
language  that  will  address  open  systems  requirements  for  electrical  as  well  as 
software  and  mechanical  interfaces. 

Global  Broadcast  System.  The  program  office  for  the  Global  Broadcast  System 
was  unsure  of  how  to  implement  the  requirement  for  determining  the  extent  of 
system  subsystem,  and  component  design  openness. 

As  program  managers  encourage  contractors  to  implement  open  systems  design 
in  system  interfaces  through  required  program  documentation,  the  program 
managers  and  contractors  can  use  this  documentation  to  determine  the  extent  of 
openness  at  all  levels,  including  between  systems,  subsystems,  and  components. 

Providing  Open  Systems  Regulation  and  Guidance  for 
Acquisition  Program  Managers 


In  addition  to  program  managers  not  documenting  a  means  for  determining  the 
extent  of  system  design  openness,  DoD  guidance  did  not  require  program 
managers  to  assess  the  impact  of  planned  system  design  openness  on  long-term 
viability  and  affordability  of  systems.  These  conditions  occurred  because  the 
Joint  Task  Force  did  not: 
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•  provide  program  managers,  in  general,  with  guidance  on  how  to 
document  the  means  for  determining  the  extent  of  system  design 
opeimess;  or 

•  establish  acquisition  policy  to  recognize  that  determining  the  level  of 
openness  of  system  design  is  most  meaningful  when  combined  with 
program  impact  assessments. 

Further,  while  executing  systems  engineering  processes,  three  of  the  four 
program  offices  visited  did  not  address  the  extent  of  system  design  openness  in 
program  design  reviews. 

Documenting  the  Means  for  Determining  the  Extent  of  Design  Opeiiness. 

The  Joint  Task  Force  did  not  provide  program  managers,  in  general,  with 
guidance  on  how  to  document  the  means  for  determining  the  extent  of  system 
design  openness.  While  the  Joint  Task  Force  had  formulated  draft  guidance  for 
program  managers  to  use  in  determining  the  extent  of  system  design  openness  in 
June  1998,  it  had  not  finalized  the  guidance.  The  Joint  Task  Force  stated  that  it 
had  not  finalized  the  guidance  because  it  was  still  validating  the  suggested 
methods  for  determining  the  extent  of  system  design  openness.  In  addition,  the 
Joint  Task  Force  stated  that  it  was  not  convinced  that  documenting  a  means  for 
determining  the  level  of  system  design  openness  added  value  unless  the  program 
managers  also  assessed  the  impact  of  the  planned  level  of  openness  on  future 
system  long-term  viability  and  affordability. 

The  Joint  Task  Force  acknowledged  that  it  needed  to  issue  guidance  in  the 
Defense  Acquisition  Deskbook  for  the  program  managers  to  use  in  determining 
the  extent  of  system  design  openness.  To  be  helpful,  the  guidance  needs  to  tell 
program  managers  how  to  structure  contract  provisions  to  provide  the  program 
office  with  visibility  and  influence  over  system  design  openness.  Program 
offices  for  the  four  programs  stated  that  the  prime  contractor  would  be  the  best 
source  for  obtaining  a  measurement  of  system  design  opeimess  but  that  their 
development  contracts  did  not  require  the  contractors  to  provide  this 
information.  Also,  the  program  offices  stated  that  performance-based 
development  contracts  offered  little  to  no  visibility  into  the  contractor’s  system 
design  configuration  during  the  development  phase  of  the  system  acquisition 
process. 

Assessing  the  Impact  of  Planned  Level  of  System  Openness.  The  Joint  Task 
Force  stated  that  it  was  not  convinced  that  determining  the  extent  of  system 
design  openness  added  value  unless  program  managers  also  assessed  die  impact 
of  a  given  level  of  system  openness  on  ftiture  system  long-term  viability  and 
affordability.  Some  system,  subsystem,  or  component  interfaces,  particularly 
those  involving  rapidly  changing  electronics,  communications,  or  computer 
technologies  can  be  far  more  critical  to  a  system’s  continued  viability  than 
slower  changing  system  interfaces.  For  example,  program  managers  for  two 
different  systems  could  each  project  that  75  percent  of  their  system’s  interfaces 
will  be  open,  but  the  systems  could  have  greatly  different  future  long-term 
viability  and  affordability  depending  on  how  fast  technology  or  requirements 
were  expected  to  change  for  the  remaining  25  percent  of  the  systems’  interfaces 
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that  were  closed.  The  Joint  Task  Force  indicated  that  program  managers  could 
provide  acquisition  decision  makers  with  a  much  more  meaningful  program 
assessment  if  they  provided  an  assessment  of  the  probable  impact  of  a  planned 
level  of  systems  openness  on  fiimre  system  long-term  viability  and  affordability 
along  widi  a  determination  of  the  extent  of  system  design  openness. 

Addressing  Open  Systems  Requirements  as  Part  of  the  Systems  Engineering 
Process.  The  Joint  Task  Force  stated  that  open  systems  guidance  should 
emphasize  implementing  open  systems  design  as  part  of  the  systems  engineering 
process.  Specifically,  the  Joint  Task  Force  stated  that  the  design  reviews  such 
as  the  preliminary  and  critical  design  reviews,  which  program  management  and 
contractor  staffs  perform  periodically  during  the  systems  engineering  process 
would  provide  a  forum  where  the  program  manager  can  address  whether  an 
open  systems  approach  is  being  successfully  applied.  These  two  design  reviews, 
as  described  in  Military  Standard  1521-B,  “Technical  Reviews  and  Audits  for 
Systems,  Equipment,  and  Computer  Software,”  June  4,  1985,  provide  an 
appropriate  forum  from  which  to  review  system  subsystems  and  components 
interfaces.  The  preliminary  design  review  is  a  formal  technical  review  of  the 
basic  design  approach  for  configuration  items.  The  critical  design  review 
follows  the  preliminary  design  review  and  verifies  whether  detail  design 
solutions  have  been  achieved.  Although  program  manager  use  of  Military 
Standard  1521-B  is  no  longer  mandatory,  DoD  Regulation  5000.2-R  emphasizes 
the  importance  of  a  structured  design  review  process  to  demonstrate  and  confirm 
completion  of  required  design  accomplishments.  Therefore,  in  order  for 
program  managers  to  determine  the  extent  of  system  openness,  they  need  to  use 
whatever  design  reviews  they  have  required  and  planned  to  make  the 
determination. 

For  the  four  program  offices  reviewed  in  detail,  only  one  program  manager 
addressed  open  systems  in  a  design  review.  For  that  system,  the  Marine  Corps 
H-1  Helicopter,  Ae  contractor  addressed  open  systems  during  its  preliminary 
design  review.  The  program  manager  indicated  that  the  contractor  would  also 
address  open  systems  during  the  critical  design  review.  If  other  program 
managers  addressed  open  systems  requirements  during  their  structured  design 
review  processes,  it  could  help  the  program  managers  emphasize,  identify,  and 
achieve  a  higher  degree  of  design  opeimess  in  the  system  development  effort. 

Benefits  of  Determining  the  Extent  and  Impact  of  System 
Design  Openness 


Without  documented  means  for  program  managers  and  contractors  to  measure 
progress  and  the  impact  of  implementing  an  open  systems  approach,  acquisition 
decision  makers  can  not  readily  gauge  how  well  program  managers  are 
achieving  the  advantages  of  using  an  open  system  design  approach.  When 
acquisition  decision  makers  assess  a  system’s  readiness  for  production,  without 
considering  the  level  of  openness  of  a  system,  they  cannot  ftilly  consider: 
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•  system  life-cycle  cost,  which  includes  technology  insertion; 

•  whether  a  system  can  accommodate  economical  technology  insertion; 
and 

•  system  affordability  to  keep  pace  with  changing  technologies. 

Given  the  long  development  cycle  for  most  major  DoD  acquisition  programs, 
systems,  subsystems  and  components  can  become  obsolete  before  systems  reach 
production.  DoD  efforts  to  keep  pace  with  technology  may  become  cost 
prohibitive  for  some  future  systems  if  an  open  systems  approach  is  not  used  in 
the  design  and  could  result  in  a  decreased  defense  capability.  Further,  program 
managers  having  knowledge  of  obsolescence  risk  throughout  the  acquisition  of  a 
system,  can  effectively  plan  for  cost-effective  systems  upgrades  throughout  the 
system’s  life. 

Recommendations 

B.l.  We  recommend  that  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  revise  DoD  Regulation  5000. 2-R,  “Mandatory 
Procedures  for  Major  Defense  Acquisition  Programs  (MDAPs)  and  Major 
Automated  Information  Systems  (MAIS)  Acquisition  Programs,”  to  require 
program  managers  to  assess  the  impact  of  projected  system  design 
openness,  including  the  readiness  and  cost  impacts  of  any  critical  closed 
proprietary  interfaces,  as  part  of  the  acquisition  strategy  used  to  support 
acquisition  milestone  reviews. 

B.2.  We  recommend  that  the  Director,  Open  Systems  Joint  Task  Force, 
update  the  Defense  Acquisition  Deskbook  to  include  guidance  to  help 
program  manager  document  the  means  for  determining  the  extent  of  system 
openness,  to  include  possible  performance  measures  for  gauging  progress  in 
implementing  an  open  systems  design  approach  as  well  as  examples  of 
possible  contract  provisions  and  strategies  that  can  be  used  to  ensure  that 
the  program  offices  acquire  the  information  needed  from  contractors  to 
measure  the  extent  of  system  design  openness. 

Management  Comments  Required 


The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  and 
his  Director,  Open  Systems  Joint  Task  Force  did  not  comment  on  a  draft  of  this 
report.  We  request  that  the  Under  Secretary  of  Defense  and  the  Director,  Open 
Systems  Joint  Task  Force  provide  comments  to  the  final  report. 


21 


Appendix  A.  Audit  Process 


Scope 

We  conducted  the  audit  from  April  1999  through  March  2000  and  reviewed 
documentation  dated  from  November  1994  through  January  2000  at  the  Office 
of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics, 
DoD  Component  Headquarters,  and  obtained  documentation  from  17  nrnjor 
Defense  acquisition  program  offices.  Specifically,  we  examined  operational 
requirements  documents,  single  acquisition  management  plans,  acquisition 
plans,  requests  for  proposals,  contract  statements  of  work,  systems  engineering 
management  plans,  and  test  and  evaluation  master  plans.  Also,  we  visited  4  of 
the  17  major  Defense  acquisition  program  offices  to  determine  whether  the 
program  offices  had  documented  a  means  for  determining  the  extent  of  design 
openness  of  systems,  subsystems,  and  components. 

DoD-wide  Corporate  Level  Government  Performance  and  Results  Act 
(GPRA)  Coverage.  In  response  to  the  GPRA,  the  Secretary  of  Defense 
aimually  establishes  DoD-wide  corporate  level  goals,  subordinate  performance 
goals,  and  performance  measures.  This  report  pertains  to  achievement  of  the 
following  goal,  subordinate  performance  goal,  and  performance  measures: 

•  FY  2000  DoD  Corporate  Level  Goal  2:  Prepare  now  for  an  uncertain 
future  by  pursuing  a  focused  modernization  effort  that  maintains  U.S. 
qualitative  superiority  in  key  warfighting  capabilities.  Transform  the 
force  by  exploiting  the  Revolution  in  Military  Affairs,  and  reengineer  the 
Department  to  achieve  a  21st  century  infrastructure.  (OO-DoD-2) 

•  FY2000  Subordinate  Performance  Goal  2.4:  Meet  combat  force’s 
needs  smarter  and  faster,  with  products  and  services  that  work  better  and 
cost  less,  by  improving  die  efficiency  of  DoD’s  acquisition  processes. 

(OO-DoD-2.4) 

•  FY  2000  Performance  Measure  2.4.1:  Major  Defense  Acquisition 
Program  (MDAP)  Cost  Growth  (In  percents).  (OO-DoD-2.4.1) 

•  FY  2000  Performance  Measure  2.4.2:  MDAP  Cycle  Time. 
(OO-DoD-2.4.2.) 

DoD  Functional  Area  Reform  Goals.  Most  major  DoD  functional  areas  have 
also  established  performance  improvement  reform  objectives  and  goals.  This 
report  pertains  to  achievement  of  the  following  acquisition  functional  issue  area 
objective  and  goal. 

Objective:  Delivering  Great  Service.  Goal:  Deliver  new  major 
Defense  systems  to  the  users  in  25  percent  less  time.  (ACQ-1.1) 
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General  Accounting  Office  High-Risk  Area.  The  General  Accounting  Office 
has  identified  several  high-risk  areas  in  the  Department  of  Defense.  This  report 
provides  coverage  of  the  Defense  weapons  system  acquisition  high-risk  area. 

Methodology 

To  evaluate  program  manager  consideration  and  use  of  open  systems  in 
developing  and  acquiring  weapon  systems,  we  evaluated  OSD  and  Military 
Department  policies  and  procedures.  We  also  examined  program  manager 
planning  and  execution  of  an  open  systems  approach  for  their  programs  as  well 
as  the  adequacy  of  DoD  acquisition  decision-maker  reviews  of  program  manager 
implementation  of  an  open  systems  approach.  We  received  technical  assistance 
in  examining  program  manager  planning  and  execution  of  an  open  systems 
approach  from  electronics  engineers  in  the  Technical  Assessment  Division  of  the 
Office  of  the  Assistant  Inspector  General  for  Audit. 

Auditing  Standards.  We  conducted  this  program  audit  in  accordance  with 
auditing  standards  issued  by  the  Comptroller  General  of  the  United  States,  as 
implemented  by  the  Inspector  General,  DoD,  and  accordingly  included  such 
tests  of  management  controls  as  we  deemed  necessary. 

Use  of  Computer-Processed  Data.  We  did  not  rely  on  computer-processed 
data  to  perform  this  audit. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  the  DoD  and  at  Defense  contractors.  Further  details  are 
available  on  request. 

Management  Control  Program 


DoD  Directive  5010.38,  “Management  Control  (MC)  Program,”  August  26, 
1996,  requires  DoD  managers  to  implement  a  comprehensive  system  of 
management  controls  that  provides  reasonable  assurance  that  programs  are 
operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  Review  of  Management  Control  Program.  In  accordance  with  DoD 
Directive  5000.1,  “Defense  Acquisition,”  March  15,  1996,  and  DoD 
Regulation  5000.2-R,  acquisition  managers  are  to  use  program  cost,  schedule, 
and  performance  parameters  as  control  objectives  to  implement  the  requirements 
of  DoD  Directive  5010.38.  Also,  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  issued  the  memorandum  “Open  Systems 
Acquisition  of  Weapon  Systems,”  July  10,  1996,  directing  that  acquisition 
milestone  decision  authorities  ensure  that  program  managers  include  open 
systems  plans  in  the  documentation  provided  to  support  acquisition  milestone 
decisions  and  that  OSD  overarching  integrated  product  teams  include  open 
systems  planning  as  a  specific  item  in  their  oversight  and  review  discussions  on 
acquisition  programs.  Accordingly,  we  limited  our  review  to  management 
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controls  directly  related  to  program  manager  consideration  and  use  of  open 
systems  in  developing  and 

acquiring  weapon  systems  and  to  overarching  integrated  product  teams  including 
open  systems  planning  as  a  specific  item  in  dieir  oversight  and  review 
discussions  on  acquisition  programs. 

Adequacy  of  Management  Controls.  We  identified  a  material  management 
control  weakness  in  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics,  as  defined  by  DoD  Instruction  5010.40, 
“Management  Control  (MC)  Program  Procedures,”  August  26,  1996.  The 
Office  of  the  Under  Secretary  had  not  implemented  effective  controls  to  ensure 
that  acquisition  milestone  decision  authorities  within  OSD  and  the  DoD 
Components  enforced  the  Under  Secretary’s  direction  for  program  managers  to 
include  open  systems  requirements  in  key  acquisition  planning  documents  as 
part  of  the  acquisition  milestone  review  process  (finding  A).  Recommendations 
A.l.  and  A. 2.,  if  implemented,  will  correct  the  material  management  control 
weakness.  A  copy  of  the  report  will  be  provided  to  the  senior  official 
responsible  for  management  controls  in  Ae  Office  of  the  Secretary  of  Defense. 

Adequacy  of  Management’s  Self  Evaluation.  The  Director,  Open  Systems 
Joint  Task  Force  conducted  a  management  control  review  that  examined  the 
adequacy  of  management  controls  to  manage  and  oversee  the  use  and 
expenditure  of  fiscal,  personnel,  and  physical  resources  assigned  to  the  Joint 
Task  Force.  The  material  management  control  weakness  we  identified  occurred 
within  the  larger  organizations  of  the  DoD  and  Component  acquisition 
executives,  and  was,  thus,  outside  the  scope  of  the  self-evaluation  the  Director, 
Open  Systems  Joint  Task  Force  performed. 

Summary  of  Prior  Coverage 


During  the  last  5  years,  the  General  Accounting  Office  issued  one  report 
relating  to  program  use  of  the  open  systems  approach. 

General  Accounting  Office,  National  Security  and  International  Affairs 
Division,  Report  99-101,  “Ballistic  Missile  Defense,  More  Common  Standards 
and  Components  Could  Result  in  Cost  Savings,”  May  21,  1999. 
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Appendix  B.  Definitions  of  Open  Systems  Terms 


The  DoD  Open  Systems  Joint  Task  Force  provided  the  following  definitions  that 
are  germane  to  understanding  the  implementation  of  an  open  systems  approach. 

Architecture.  Architecture  is  the  organizational  structure  of  a  system  or 
component,  the  relationships,  principles  and  guidelines  governing  design  and 
evolution  over  time. 

Commercial  Item.  A  commercial  item  is  any  item  other  than  real  property  that 
is  of  a  type  customarily  used  for  nongovernmental  purposes  and  that  has  been 
sold  to  the  general  public  or  offered  for  sale  to  the  general  public. 

Closed  Interfaces.  Closed  interfaces  are  privately  controlled  system  and 
subsystem  boundary  descriptions  for  interfaces  that  are  not  disclosed  to  the 
public  or  are  unique  to  a  single  supplier. 

Interface  Standard.  An  interface  standard  specifies  the  physical  or  functional 
interface  characteristics  of  systems,  subsystems,  equipment,  assemblies, 
components,  items  or  parts  to  permit  interchangeability,  interconnection, 
interoperability,  compatibility,  or  communications. 

Interoperability.  Interoperability  is  the  ability  of  two  or  more  systems  or 
components  to  exchange  data  and  use  information. 

Joint  Technical  Architecture.  The  Joint  Technical  Architecture  defines  the 
DoD  minimum  set  of  rules  governing  the  arrangement,  interaction,  and 
interdependence  of  the  parts  or  elements,  whose  purpose  is  to  ensure  that 
systems  conform  to  a  specific  set  of  requirements.  It  identifies  system  services, 
interfaces,  standards,  and  the  relationships. 

Level  of  Openness.  The  level  of  openness  is  the  system,  subsystem,  or 
component  level  at  which  the  interfaces  conform  to  open  standards.  The 
contractor  or  supplier  may  control  design,  interfaces,  repair,  and 
implementation  below  the  level  of  openness.  The  level  of  openness  will  affect 
the  overall  performance,  life-cycle  costs,  long-term  supportability,  acquisition 
cycle  time,  interoperability,  intra-operability,  ease  of  technology  insertion,  and 
the  extent  of  organic  repair  of  a  system. 

Modular.  Modular  is  the  design  concept  in  which  interchangeable  units  are 
used  to  create  a  functional  end  product. 

Module.  A  module  is  an  interchangeable  item  that  contains  components.  In 
computer  programming,  a  program  unit  that  is  discrete  and  identifiable  with 
respect  to  compiling,  combining  with  other  modules  and  loading  is  called  a 
module. 


25 


Nondevelopmental  Item.  A  nondevelopmental  item  is  any  previously 
developed  item  of  supply  used  exclusively  for  governmental  purposes  by  a 
Federal  agency,  a  State  or  local  government,  or  a  foreign  government  with 
which  the  United  States  has  a  mutual  defense  cooperation  agreement. 

Open  Specifications.  Open  specifications  are  public  specifications  maintained 
by  an  open,  public  consensus  process  to  accommodate  new  technologies  over 
time  and  consistent  with  international  standards. 

Open  Standards.  Open  standards  are  widely  accepted  and  supported  standards 
set  by  recognized  standards  organizations  or  the  commercial  market  place.  Open 
standards  support  interoperability,  portability,  and  scalability  and  are  equally 
available  to  tiie  general  public  at  no  cost  or  with  a  moderate  license  fee. 

Open  System.  An  open  system  is  a  system  that  implements  sufficient  open 
standards  for  interfaces,  services,  and  supporting  formats  to  enable  properly 
engineered  components  to  be  used  across  a  wide  range  of  systems  with  minimal 
changes,  to  interoperate  with  other  components  on  local  and  remote  systems, 
and  to  interact  with  users  in  a  style  that  facilitates  portability.  An  open  system 
is  characterized  by  the  following: 

•  well  defined,  widely  used,  preferably  nonproprietary  interfaces  and 
protocols; 

•  use  of  standards  which  are  developed  and  adopted  by  recognized  standards 
bodies  or  the  commercial  market  place; 

•  definition  of  all  aspects  of  system  interfaces  to  facilitate  new  or  additional 
systems  capabilities  for  a  wide  range  of  applications;  and 

•  explicit  provision  for  expansion  or  upgrading  through  the  incorporation  of 
additional  or  higher  performance  elements  with  minimal  impact  on  the 
system. 

Open  Systems  Approach.  An  open  systems  approach  is  an  integrated  business 
and  technical  strategy  to  choose  commercially  supported  specifications  and 
standards  for  selected  system  interfaces  (external,  internal,  functional,  and 
physical),  products,  practices,  and  tools,  and  build  systems  based  on  modular 
hardware  and  software  design.  Program  selection  of  commercial  specifications 
and  standards  is  based  on: 

•  those  standards  that  industry  standards  bodies  have  adapted  or  are  industry 
de  facto  standards  (those  successful  in  the  market  place); 

•  market  research  that  evaluates  the  short  and  long-term  availability  of 
products; 

•  a  disciplined  systems  engineering  process  that  examines  tradeoffs  of 
performance; 
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•  supportability  and  upgrade  potential  within  defined  cost  constraint;  and 

•  allowance  for  continued  access  to  technological  innovation  supported  by 
many  customers  and  a  broad  industrial  base. 

Open  Systems  Architecture.  An  open  systems  architecture  is  a  system 
architecture  produced  by  an  open  systems  approach  and  using  open  systems 
specifications  and  standards  to  an  appropriate  level. 

Open  Systems  Strategy.  An  open  systems  strategy  focuses  on  fielding  superior 
warfighting  capability  more  quickly  and  more  affordably  by  using  multiple 
suppliers  and  commercially  supported  practices,  products,  specifications,  and 
standards,  which  are  selected  based  on  performance,  cost,  industry  acceptance, 
long-term  availability  and  supportability,  and  upgrade  potential. 

Portability.  Portability  is  the  ease  with  which  a  system,  component,  data,  or 
user  can  be  transferred  from  one  hardware  or  software  environment  to  another. 

Proprietary  Specifications.  Proprietary  specifications  are  exclusively  owned 
by  a  private  individual  or  corporation  under  a  trademark  or  patent,  the  use  of 
which  would  require  a  license. 

Scalability.  Scalability  is  the  capability  to  adapt  hardware  or  software  to 
accommodate  changing  workloads. 

Specification.  A  specification  is  a  document  that  prescribes,  in  a  complete, 
precise,  verifiable  manner,  the  requirements,  design,  behavior,  or 
characteristics  of  a  system  or  system  component. 

Standard.  A  standard  is  a  document  that  establishes  uniform  engineering  and 
technical  requirements  for  processes,  procedures,  practices,  and  methods. 
Standards  may  also  establish  requirements  for  selection,  application,  and  design 
criteria  of  material. 

System  Architecture.  A  system  architecture  is  a  description,  including 
graphics,  of  systems  and  interconnections  providing  for  or  supporting 
warfighting  functions.  The  system  architecture  defines  the  physical  connection, 
location,  and  identification  of  the  key  nodes,  circuits,  networks,  and  warfighting 
platforms  and  specifies  system  and  component  performance  parameters.  It  is 
constructed  to  satisfy  operational  architecture  requirements  per  standards 
defined  in  the  Joint  Technical  Architecture.  The  system  architecture  shows  how 
multiple  systems  within  a  subject  area  link  and  interoperate,  and  may  describe 
the  internal  construction  or  operations  of  particular  systems  within  the 
architecture. 
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Appendix  C.  Public  Law  and  Government  Policy 


Public  law  and  implementing  Federal  and  Government  acquisition  policies 
mandate  that  program  managers  use  an  open  systems  approach  in  &e  weapon 
system  acquisition  process. 

Public  Law 


Section  12(d)  of  Public  Law  104-113,  “National  Technology  Transfer  and 
Advancement  Act  of  1995,”  March  7,  1996,  requires  that  all  Federal  agencies 
and  departments  use  technical  standards  that  are  developed  or  adopted  by 
voluntary  consensus  standards  bodies,  using  such  technical  standards  as  a  means 
to  carry  out  policy  objectives  or  activities. 

Government  Policy 

Office  of  Management  and  Budget.  The  Office  of  Management  and  Budget 
issued  Circular  A-119,  “Federal  Participation  in  the  Development  and  Use  of 
Voluntary  Consensus  Standards  and  in  Conformity  Assessment  Activities,” 
February  10,  1998.  This  Circular  directs  agencies  to  use  voluntary  consensus 
standards  in  lieu  of  government-unique  standards  except  where  inconsistent  with 
law  or  otherwise  impractical.  The  policies  in  the  Circular  are  intended  to 
reduce  agency  reliance  on  government-unique  standards. 

Department  of  Defense.  The  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  mandated  program  manager  use  of  an  open  systems  approach  in  his 
November  29,  1994  memorandum,  “Acquisition  of  Weapons  Systems 
Electronics  Using  Open  Systems  Specifications  and  Standards.”  The 
memorandum  required  acquisition  program  mangers  to  use  open  systems 
specifications  (electrical,  mechanical,  thermal)  for  acquisition  of  weapon 
systems  electronics  to  the  greatest  extent  practical.  The  memorandum  further 
stated  that  acquisition  program  managers  should  design,  develop,  and  construct 
systems  and  subsystems  as  open  systems  during  the  acquisition  and  modification 
process  to  reduce  life-cycle  cost  and  facilitate  effective  weapon  system  intra  and 
interoperability.  On  March  15,  1996,  the  Under  Secretary  expanded  an  open 
systems  requirement  to  include  all  system  elements  (mechanical,  electrical,  and 
software)  in  developing  weapon  systems. 


28 


Appendix  D.  Open  Systems  Joint  Task  Force 

Education  and  Outreach  Initiatives 


Since  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
chartered  the  Joint  Task  Force  in  November  1994,  it  has  made  substantial  efforts  to 
promote  program  manager  use  of  an  open  systems  approach  in  weapon  system 
acquisition.  The  Joint  Task  Force’s  efforts  have  emphasized  education  and  outreach  to 
the  acquisition  conununity. 

Education 


The  Joint  Task  Force  developed  educational  products  on  the  use  of  open  systems 
and  made  these  products  available  to  the  DoD  acquisition  workforce.  The  Joint 
Task  Force  educational  products  include  a  3-hour  computer-based  open  systems 
tutorial  course  available  on  CD-ROM,  an  open  system  implementation  guide,  a 
weapon  system  case  study,  an  open  systems  engineering  tutorial,  and  various 
papers,  zuticles,  and  brochures.  To  broaden  usage  of  the  educational  products, 
the  Joint  Task  Force  has  made  them  available  on  an  open  systems  web  site. 
Additionally,  the  Joint  Task  Force  provided  the  Defense  Acquisition  University 
input  to  update  acquisition  courses  to  better  cover  program  manager  use  of  an 
open  system  approach  in  developing  and  acquiring  weapon  systems. 

Specifically,  the  Joint  Task  Force  collaborated  with  the  Defense  Acquisition 
University  to  modify  segments  of  the  following  courses  to  include  open  systems 
concepts  and  principles: 

•  Acquisition  Program  Management  Course  Open  Systems  Program 
Management  Elective 

•  Acquisition  Program  Management  Open  Systems  Engineering  Elective 

•  Communications  20 land  301 

•  Contracting  301  Seminar 

•  Executive  Program  Management  Course 

•  Software  Acquisition  Management  20 land  301 

•  Systems  Acquisition  for  Contracting  Personnel 

•  Systems  Acquisition  Management  101  and  201 

•  Systems  Engineering  201  and  301 

•  Test  and  Evaluation  301 

Outreach  to  the  Acquisition  Community 

open  systems  Joint  Task  Force  outreach  efforts  have  included  performing 
assessments  of  program  manager  implementation  of  an  open  systems  approach 
in  weapon  systems  acquisition,  participating  in  acquisition  program  integrated 
product  teams,  providing  funding  to  open  system  projects  within  the  DoD 
components,  participating  in  conferences,  providing  briefings  to  industry  and 
professional  groups,  and  providing  acquisition  planning  document  templates  and 
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lessons  learned  regarding  program  manager  implementation  of  the  open  system 
approach  to  program  managers  requesting  assistance.  Details  of  these  efforts 
include: 

•  Acquisition  Program  Assessments.  In  January  1996,  the  Joint  Task  Force 
examined  the  DoD  acquisition  programs  to  ensure  that  program  managers 
were  effectively  implementing  the  directive  on  program  manager  use  of  an 
open  systems  approach  in  developing  and  acquiring  weapon  systems.  Since 
January  1996,  the  Joint  Task  Force  had  performed  and  reported  on 
assessments  of  open  system  implementation  for  five  major  Defense 
acquisition  programs.  An  open  systems  Joint  Task  Force  had  also  assessed 
program  implementation  of  open  systems  for  in  five  additional  programs  but 
did  not  prepare  formal  reports  of  its  assessments. 

•  Participation  on  Integrated  Products  Teams.  The  Joint  Task  Force  has 
participated  in  more  than  25  integrated  product  teams  to  enhance  program 
manager  use  of  open  systems  in  weapon  systems  development.  In  FY  1999, 
the  Joint  Task  Force  participated  on  integrated  product  teams  for  the 
C-130  Upgrade,  C-17  aircraft.  Crusader,  F-22,  and  Surface  Combatant  for 
21st  Cenmry. 

•  Funding  Open  Systems  Projects.  In  February  1996,  the  Joint  Task  Force 
provided  funding  to  the  two  projects:  the  Navy’s  AV-8B  Aircraft  Open 
Systems  Core  Avionics  and  &e  Air  Force’s  F-15E  Multipurpose  Display 
Processor  that  the  Principal  Deputy  Under  Secretary  of  Defense  for 
Acquisition  and  Technology  designated  as  pilot  programs  for  implementing 
an  open  systems  approach  within  DoD.  The  Joint  Task  Force  also  provided 
funding  to  assist  the  program  managers  for  three  additional  programs  to 
develop  an  open  systems  approach. 

•  Conferences  with  Industry  and  Professional  Groups.  The  Joint  Task 
Force  has  participated  in  21  conferences  with  industry  and  professional 
groups.  The  Joint  Task  Force  participation  included  presenting  papers  on 
open  systems  architecture  and  setting  up  booths  to  distribute  information  on 
the  open  system  initiative.  Additionally,  the  Joint  Task  Force  held 
numerous  briefings  with  industry  and  professional  societies  on  current  DoD 
efforts  to  implement  open  systems  and  directed  dialog  with  industry  to 
encourage  technical  discussions  on  how  program  mangers  use  of  open 
systems  effects  business  opportunities  with  &e  DoD. 

•  Templates  and  Lessons  Learned.  The  Joint  Task  Force  provided 
acquisition  planning  document  templates  to  selected  program  offices  to  help 
acquisition  staffs  with  the  appropriate  inclusion  of  open  systems  language  in 
single  acquisition  management  plans  and  system  engineering  management 
plans.  Additionally,  the  Joint  Task  Force  provided  the  DoD  acquisition 
community  with  lessons  learned  on  program  implementation  of  an  open 
systems  approach.  The  lessons  learned  included  case  studies  fiilly  analyzing 
and  documenting  examples  of  open  system  technical  and  business  strategies 
that  acquisition  program  managers  have  used.  The  Joint  Task  Force  also 
uses  the  lessons  learned  in  preparing  educational  offerings. 
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Appendix  E.  Inclusion  of  Open  Systems  Objectives  and  Requirements  in 

Acquisition  Planning  Documents 


Footnotes  are  explained  at  the  end  of  this  appendix. 
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Footnotes  are  explained  at  the  end  of  this  appendix. 


‘Legacy  acquisition  program. 

^New  acquisition  program. 

^Totals  include  oidy  Aose  programs  that  were  required  to  prepare  each  acquisition  planning  document. 
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m  the  finding  that  t>^:re  ia  no 

or  qQ'mmm  to  evalii-ate  and  determine-  the  degree 
oS  0|3i80Bi^i-S  for  development-  Me  support  the 

:rec?0-iiiilT.^n4li.t  ion  to  develop  goidelineo  abd  tequitd  the  0^f 
to  esaese  the  iopaot  of  ayateft^ 
ailld-  oponnt^^  m  port  of  Acquisition  Kile'S-tone- 
Idig-^lly,  if  ove:^  program  follo^^  thla  o|Mf-ii  ayatem 
appro^^ch,  then  prohlens  intefrdtifig  proda^^ta.  iSis  -Joint 

Ifadin  Ey-steno  (dtRS^  or  Eitbed-'Jad  Battle 
gpn^bdVForoe  :xxi  2attl«  CotmsM  Brigade  a.M  Mlc¥ 
(iSCXFSCBS)  would  be  llo^^ever,.  m  at^t<?d  i^n  page 

1-D  of  the  draft  report,  there  m  tQ  validate 

•E-uch  programs  h^rd«ar«s;  or  software 

a.:£:t:hitecture .  If  this  op^P  r/tt^m  approach  we:re  utilized, 
tKe-n  eoftKSre  aod  hardv^-iri©  would  increase,  fhe 

eruaader  system  ^dll.  'tew  ap^n  synten  arrhiterture - 
}5owever*  if  the  products  we  directed  to  use  do  not  have 
the  desired  degree  of  c^pspness.,  we  nay  have  to  auhoptinire- 
ours,  ib  effect,  mtd  soB^sone  to  iia-fce  the  law  but  at 
the  same  tim  eosnaone  -etso  to  enfcrce  it. 

tM  WU  has  adopted  the  Comoon  Operatln^^  Bjivirc-nifient  lOT) 
opneept  in  the  Pll  tm,  with  the  Glohtfel  Centred 

.system  COB  as  its  first  itt^lem.ebcat ion.  (referenced 

if^  i^n^rsion  .3..  1  of  the  doint  Terluiioel.  Arcllit€€tyre  1.  Jf  A) )  < 
"This  cos  lays  the  foundation  for  the  provlalOE  cf 
gtepd^rdiz^nd^  ctoiom  services.  It  is  described  a..a  a 
.iOft*wsre  architecture,  an  approach  for  building 
interoperable  sySteitsSi  a  colleotiOh  Of  rsusable  HOttw^rc 
^components,  a  software  infrsi.sttM.ctoret  B.;nd  of 

guidelines  and  standards.  n>ain  enphiiji-?  it  this 

^ve.rsioti  -of  the  A^my  fechtic^i^l  Architi^ctorij'.  (ata^  is 
ntilitlr>g  the  COB  concept >>  softw'^re  architecture-,  and 
buildihg'to  a  staitdard  layer  of  Application  Program 
Interfaces  (Af la^ , 

Tl^  Crusher  eyettm  ae  im^ceing  the  reguire-cient  on  the 
ptl^  COntra<;tor  to  ^npl^inri^nt  ^  Coirirond.  control, 
<X")mmuniCi£tions,  computers  &  intelligence  open  syst^em 

architecture  to  efficiently  share  information  be-tvii^en  its 
uany  corrputerized  subsystems  -  Csrassder  ^  s  On-board  CO-mputsr 
resources  Will  bs  capable  of  pstforifdbg  t-schrilcal  and 
tactical  firs  eontrol*  dsclsl-on  aids*  posit i<5iir^  aM 
navigation  -coMr-ol,  control,  mmminim  handling 

Dontrcl,  Self  dsfsnae  system,  :rssyppiy  activities, 
ecTOunicatlot  pr-octaair^,  diagrc^atica/prognontica, 
flkirtenancc  and  pptrntnr/msaintw-anco  training. 
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of  Mgitsil 

€ami.^iCli..tl««  ;pc^si.ti&jiipg  syatem  t  digital 

inapfping  rapabiliticis^  a;nd  arr:ert3i:n^  battle! 
ideHti£ir^iticn  ayeiem  ^BCT^l  will  CnisindaT  a  clear 

picatm^  ■f^f'  tM  battle  gieM  arid  enhance  rriissicn  cape'biliti^a 
fchxcagh.  increased  situati-C»i  <^f 

fratricide^  taatet  tirri^ss  and  increased  accuracy  of 

:t!Otll  indirect  fires.  The  ocn-nunicat ion 

subaystc^'  that  enables  ^uaader  tO'  lihk 

battlefield  glutlotnB  uill  mMnim  th^  vm  Qt  Tranamissicn 
Cbntrbl  Prot^SOOl-I^t^irhfst  Fratcccl  !(TCP-IF[  corrrercial 
^i^twoilcing  techneiogier,  Sifcld^i^iRS  syst^irr: 
pregrarri  •  <SI  transcei’/^sr^^  BIP  Int^il^^t 

OTtitroll^'t  NIL«&TQ-lSS-22.DXt  renbat  net  radio 

(OTRI.I  p'rotoDcl.  The  cwibinaticn  of  tha  SIP^  %UCm 

and  HlL-£tD-liS~220A  Mill  Cifc’iisadgr  to  have  Ecamless 

pachjet  erHltehing,  routing  and  relaying  of  iressages  bsti^een. 
timt  pron^?:sscrs  and  arrmg  other  Crusader  ele^'se.ntSj  Che  FOC 
and  gntentially  other  t^roucfid  ynit^. 

In  or-iier  to  ^  true  open  syrten.,  we  approach  the 

problem  fron  both  software  and  hardi^‘are  p^rsp-ectlv^giS^ ? 
are  goin^'  to  enaure  tM  proc^^a^^lng  r@sources^  for  electronic 
&re11l.-lt*^Ct;:yrB  not  uniqi^e  to  an  applicatiDn. 

Additionally,  xe  will  ensure  the  mmmcm 

efacternal  device  re^i0urc«8  avaitablR.  iron  the  nain 
proseesing  umt^-  Cur  '»cpm  system  architecture"  uses 
mwtmn  operatirjg  systen  softx^re^  vell-rmt^n  ardi,  SCt^pced 
data-bjs  at-andards^  microproceSig4>r  iri§  and 

pewsr  distribution,  and  eontroX  elonsnt  technol-ogtes.  The 
architecture  provides  the  electrcai.ca  .resourcer  needed  to 
aoDcnplish  the  cvsrall  Cruaii-der  ntissiOh^  with 
tebysthes^  to  support  tho  tranisition  of  the  technology  fren 
tbg:  devolopiffint  cycle  of  the  program  to  production. 

Tm  srot4ti|<5turo  is  a  nc-dular, 

hiFrarohical,  layered  architecture  hared  cn  open  standards 
for  .its  interfaces  ..  Me  ars  developing  the  CruS-adsr 
software  to  be  im^Q^rmrit  of  h^^^t  platiorn  hardwtro. 
mt  unlguo  to  other  software  applicaticar  in  the  ayrten 
and,  for  the  nost  part.,  not  iin.i!|n,e  to  Syat^im  it\  Which 
it  reeide^.  The  ulii^i^te  cbje-CtU'^  ia  Icwsr ing  iLtc^oycle 
CCiFit  ty'  .having  software  that  is  resilient  to  changer  in 
regasienentej  l^rdware  upgrades  r  Oper'iatitjg 
soldier  -^^ohln^  , 
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